Papers with general language understanding
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation (2024.acl-long)
Copied to clipboard
| Challenge: | Weight quantization has emerged as a popular solution to reduce memory and computational demands. |
| Approach: | They propose a framework that synergizes Quantization-Aware Training (QAT) with Knowledge Distillation (KD) to boost the performance of LLMs at sub-4-bit. |
| Outcome: | The proposed framework outperforms existing QAT methods on language understanding and complex reasoning benchmarks on sub-4-bit models. |
Auto prompting without training labels: An LLM cascade for product quality assessment in e-commerce catalogs (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Our system generates and refines prompts for evaluating attribute quality across tens of thousands of product category–attribute pairs. |
| Approach: | They propose a free cascade for auto-prompting Large Language Models (LLMs) that generates and refines prompts for evaluating attribute quality across tens of thousands of product category–attribute pairs. |
| Outcome: | The proposed system improves precision and recall by 8–10% over chain-of-thought prompting while reducing domain expert effort from 5.1 hours to 3 minutes per attribute. |
Task Calibration: Calibrating Large Language Models on Inference Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive zero-shot performance on inference tasks, however, they may suffer from spurious correlations between input texts and output labels, which limits their ability to reason based purely on general language understanding. |
| Approach: | They propose a zero-shot and inference-only calibration method inspired by mutual information which recovers LLM performance through task reformulation. |
| Outcome: | The proposed calibration method improves on 13 benchmarks and prompt templates and can be integrated with other calibration methods. |
IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning. |
| Approach: | They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset. |
| Outcome: | The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC. |
SocioProbe: What, When, and Where Language Models Learn about Sociodemographics (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have outperformed other models on a wide range of tasks . however, there is still little understanding of their knowledge of higher-level aspects of language . |
| Approach: | They investigate whether pre-trained language models have knowledge of sociodemographics . they use traditional probing techniques to probe the knowledge of single-GPU PLMs based on multiple English data sets . |
| Outcome: | The results show that pre-trained language models outperform other models on a wide range of tasks. |
Bias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Dataset (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models exhibit social biases . however, debiasing methods may degrade the capabilities of LLMs if they are not properly evaluated . |
| Approach: | They propose a Japanese benchmark to evaluate social biases and cultural commonsense in large language models in a unified format. |
| Outcome: | The proposed method degrades the performance of the LLMs on the cultural commonsense task by 75%. |
SoftMCL: Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for pre-training language models capture general language understanding but fail to distinguish affective impact of a particular context to a specific word. |
| Approach: | They propose a soft momentum contrastive learning method for fine-grained sentiment-aware pre-training that uses valence ratings as soft-label supervision instead of hard labels. |
| Outcome: | The proposed method improves on four sentiment-related tasks and the results are published online. |